1. Technical Field
The present application relates generally to an improved data processing system and method. More specifically, the present application is directed to using an Adaptive List Prefetch in order to improve the performance of skip-sequential processing of data.
2. Description of Related Art
A host processor is connected or attached to a storage controller which manages a set of non-volatile storage devices. The non-volatile storage devices are generally spinning disks such as hard disk drives. The storage controller also contains a high-speed cache. Data is read from the cache into the host via a high-speed fiber channel. Disks are generally much slower than the channels, largely because of the time to reposition the disk each time that a new set of data needs to be read. The time to reposition the device is usually called “seek and latency”. “Latency time” is a function of the disk rotation speed, and “seek time” involves the mechanical motion of an arm along the radius of the disk.
When an application or database system needs to read data sequentially, a technique known as prestaging (also known sometimes as prefetching) is used to minimize the effects of disk repositioning. Prestaging brings a large chunk of data in the cache, so that less time is spent waiting for seek and latency time; that is, the disk needs to be repositioned less frequently. Furthermore, in cases where the data is striped across multiple disks, such as in a Redundant Array of Independent Drives (or Disks) (RAID) array, prestaging also makes it possible to sequentially read each disk in the array at the same time, transparent to the application. In other words, although the application may serially be read data from the cache, large chunks of data can be simultaneously read from multiple disks into the cache.
Prestaging may be initiated in a number of ways. In some systems, the host processor may request prestaging. Alternatively the storage controller may employ a sequential detection algorithm and choose on its own to employ prestaging. The host processor generally makes its decision based on foresight about what the application is going to do. In the latter case, the storage controller makes its decision based on hindsight about what the application has been doing.
However, there are applications whose data reference patterns are known as “skip-sequential” where prestaging is not done, but where prestaging might be beneficial. Skip-sequential indicates that the records are not-contiguous within the storage device but are relatively close together based on the addresses associated with the records. Whether or not prestaging is beneficial for skip-sequential patterns depends on the clustering of the records. For example, if the application needs to read every other record, it is faster to read every record than it is to try to “skip” every other record.
One known solution for skipping records is a count-key-data (CKD) solution that requires help from the host but is not supported by SCSI architecture. The CKD channel command word (CCW) architecture supports a SKIP flag. On modern storage controllers that emulate CKD architecture, the SKIP flag has to be emulated. Such controllers will read a “skipped” record from the disk into cache, but will not send the record across the channel. Nevertheless, the SKIP flag is not supported by the other systems, such as z/OS Media Manager. In any case, to use the SKIP flag requires a CCW for every skipped record, and on today's fast Fiber Connectivity (FICON) channels, every CCW constitutes significant overhead. So, the old SKIP flag architecture is not very efficient.
Another CKD method of skipping records is to generate one Locate Record CCW each time that some records need to be skipped. From the storage controller viewpoint, each Locate Record CCW is treated as if it were a distinct I/O from the previous Locate Record CCW. For example, suppose that a channel program consists of two Locate Record CCWs, and each Locate Record CCW is followed by a Read Data CCW. Further assume that none of the data is in cache. The first Locate Record CCW tells the storage controller which address is going to be read and the following Read Data CCW causes the storage controller to make a request to the physical disk in order to seek to the appropriate track, wait for the disk to position itself to the appropriate record, and then read the record into the controller's cache. As the data is being read into cache, the record may also be sent back to the host. When the host has received the data, the next Locate Record CCW will be sent by the host channel to the storage controller. Performing such operations takes time. During this time, a fast disk may skip past the address specified in the second Locate Record. If that happens, the application must wait for another rotation of the disk. This extra rotation of the device adds to the elapsed time of a skip-sequential application. If the application reads every other record, or every third record, it would have been faster to read every record in order to avoid these lost rotations.
Parallelism gives rise to another reason why it may be faster to read every record than to skip some records. Many RAID device implementations stripe CKD volumes across many disks. Striping a volume is similar to a table in a document, where a disk is a column and a stripe is one of the entries in the table. A stripe includes all of the entries in one row. Often eight disks make up the disk array. When reading sequentially, it is optimal to employ all eight disks at the same time. Theoretically a skip-sequential application could break up Locate Record “domains” into separate reads and writes or input/output operations (I/Os) and execute them in parallel. This would potentially employ every disk in the RAID array, but at the same time increasing the number of I/Os adds overhead to the host system.